Automatic SignWriting Recognition: Combining Machine Learning and Expert Knowledge to Solve a Novel Problem

Sign languages are viso-gestual languages, using space and movement to convey meaning. To be able transcribe them, SignWriting uses an iconic system of symbols meaningfully arranged in the page. This two-dimensional system, however, is very different traditional writing systems, so its automatic processing poses a novel challenge for computational linguistics. In this article, we present problem state art artificial intelligence: recognition. We examine problem, model underlying data domain, first solution form expert that exploits domain knowledge encoded modelization. adaptable pipeline neural networks deterministic processing, overcoming challenges posed by novelty originality problem. Thanks our modelization, it improves accuracy compared straight-forward deep learning approach 17%. All code publicly available, may useful not only but also other similar graphical data.